Signal processing apparatus and method, noise reduction apparatus and method, and program therefor

ABSTRACT

A signal processing apparatus including: a first noise reduction processing means that performs first noise reduction processing on an image, in which each of multitudes of pixels has one of a plurality of color components and the color components are distributed regularly, based only on pixel arrangement to obtain a first processed image; a color component separation means that separates the first processed image into each of the color components to obtain a plurality of color component images; and a signal classification means that compares a signal value of a target pixel for processing with a signal value of each pixel included in a predetermined range of area around the target pixel and classifies each pixel within the predetermined range of area into one of a plurality of groups based on the comparison result.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a signal processing apparatus andmethod for classifying an image into a plurality of groups according tothe magnitude of signal values of the image, a noise reduction apparatusand method for performing noise reduction processing, and a computerprogram product for causing a computer to perform the signal processingmethod and noise reduction method.

2. Description of the Related Art

Pixel density and sensitivity of CCD's used in digital cameras and thelike are ever increasing. The increase in the sensitivity of CCD's,however, has given rise to a problem of noise included in imagesobtained by photographing. Consequently, various methods have beenproposed for reducing noise included in images. For example, a noisereduction method in which an amount of noise in each pixel of an imageis estimated, as well as imaging conditions, then the estimated mount ofnoise is corrected according to the imaging conditions, and reducing thenoise in the image based on the corrected amount of noise is proposed asdescribed, for example, in U.S. Patent Application Publication No.20050099515. Another method is also proposed as described, for example,in U.S. Patent Application Publication No. 20060039039, in which CCD-RAWdata outputted from a CCD are separated into each of R, G, B colorcomponents, and noise reduction processing is performed whilemaintaining the correlation between each color component.

The method described in U.S. Patent Application Publication No.20050099515 reduces the noise only by smoothing processing, though noiseamount is estimated with respect to each pixel, so that the method couldnot reduce the noise appropriately when the amount of noise is large. Inthe mean time, the method described in U.S. Patent ApplicationPublication No. 20060039039 could cause an image to be blurred when thephotographing sensitivity is increased and hence the noise amounts to alarge amount, since in such a case, distinction between the noise andsignals of edges and the like is difficult or the correlation betweeneach color component becomes small.

SUMMARY OF THE INVENTION

The present invention has been developed in view of the circumstancesdescribed above, and it is an object of the present invention to enableappropriate noise reduction, particularly when a large amount of noiseis present.

A signal processing apparatus according to the present inventionincludes:

a first noise reduction processing means that performs first noisereduction processing on an image, in which each of multitudes of pixelshas one of a plurality of color components and the color components aredistributed regularly, based only on pixel arrangement to obtain a firstprocessed image;

a color component separation means that separates the first processedimage into each of the color components to obtain a plurality of colorcomponent images; and

a signal classification means that compares a signal value of a targetpixel for processing with a signal value of each pixel included in apredetermined range of area around the target pixel and classifies eachpixel within the predetermined range of area into one of a plurality ofgroups based on the comparison result.

Here, the processing target image in the present invention is a CCD-RAWimage represented by raw data, so-called CCD-RAW data, outputted from animaging device, in which each of multitudes of pixels has one of aplurality of color components and the color components are distributedregularly. Thus, the color component of a certain pixel may possibly bedifferent from that of a pixel around the certain pixel. The term “firstnoise reduction processing based only on pixel arrangement” as usedherein means to simply perform noise reduction processing based only ona signal value of a noise reduction target pixel and a signal value of apixel around the target pixel without regarding the distribution of thecolor components.

Here, each of the color component images may be classified into aplurality of groups based on the comparison result without performingthe first noise reduction processing. However, if a large amount ofnoise is contained in an image in which each of multitudes of pixels hasone of a plurality of color components and the color components aredistributed regularly, that is, in a raw image so-called CCD-RAW imageoutputted from an imaging device, a comparison result between a signalvalue of a target pixel for processing and a signal value of a pixelincluded in a predetermined range of area around the target image ineach of the color component images is degraded due to the noise, so thateach of the color component images can not be accurately classified intoa plurality of groups. In particular, a CCD-RAW image obtained by highsensitivity photographing contains a great amount of noise, which makesthe classification even more difficult.

In the present invention, the first noise reduction processing isperformed on an image, in which each of multitudes of pixels has one ofa plurality of color components, and the color components aredistributed regularly, i.e., a raw image so-called CCD-RAW image, basedonly on pixel arrangement. This allows noise in each of the colorcomponent images obtained from a CCD-RAW image may be reduced even whena large amount noise is contained in the CCD-RAW image as in highsensitivity photographing. Consequently, each of the color componentimages may be classified accurately into a plurality of groups accordingto the comparison result without being affected by the noise.

Further, noise reduction processing may be performed appropriatelyaccording to the classified groups, so that a high quality image withreduced noise may be obtained.

Still further, the first noise reduction is performed based only onpixel arrangement, i.e., without regarding the distribution of the colorcomponents, so that the first noise reduction processing is performed,in effect, in the frequency band greater than or equal to Nyquistfrequency of each of the color components included in an image.Consequently, an image subjected to the noise reduction processing maypossibly be blurred slightly, but the blur is reduced in each of thecolor component images obtained by separating the image into each of thecolor components. This allows each of the color component images to beclassified into a plurality of groups according to the comparison resultwithout being affected by the blur.

In the signal processing apparatus according to the present invention,the signal classification means may be a means that classifies eachpixel within the predetermined range of area into a flat region grouphaving a relatively small variance of the signal values or a signalregion group having a relatively large variance of the signal valuesaccording to the comparison result.

In the signal processing apparatus according to the present invention,the first noise reduction processing may be filtering processing by alow-pass filter on each of the pixels in the image.

Further, the signal processing apparatus according to the presentinvention may further includes a second noise reduction processing meansthat performs second noise reduction processing on each of the colorcomponent images according to a variance direction of the signal valuesin each of the pixels in each of the color component images; and thesignal classification means may be a means that performs the signalclassification processing on each of the color component imagessubjected to the second noise reduction processing.

As for the “variation direction of the signal values”, for example, fourdirections of horizontal, vertical, upper left to lower right, and lowerleft to upper right with reference to the target pixel may be used.

In this case, the second noise reduction means may be a means thatperforms the second noise reduction processing by performing filteringprocessing on each of the pixels in each of the color component imagesin a plurality of predetermined directions by a high-pass filter todetect a direction in which the variance of the signal values issmallest based on the filtering result, and performing filtering by alow-pass filter on the direction in which the variance of the signalvalues is smallest.

This may reduce noise in an image without blurring an edge or the like,since the filtering processing by the low-pass filter is not performedin a direction that crosses the edge or the like where signal valuesvaries.

In this case, the second noise reduction means may be a means thatperforms the second noise reduction processing by performing filteringprocessing by a plurality of types of high-pass filters having differentfrequency characteristics from each other on each of the pixels in eachof the color component images in a plurality of predetermined directionsto obtain a plurality of filtering results, detecting a direction inwhich the variance of the signal values is smallest based on theplurality of filtering results, and performing filtering by a low-passfilter on the direction in which the variance of the signal values issmallest.

Here, signal values in an image may vary gradually like gradation,abruptly like an edge, or finely like a pattern on clothing, andfrequency characteristics differ in each case. Consequently, byperforming filtering processing by a plurality of types of high-passfilters having different frequency characteristics from each other toobtain a plurality of filtering results and detecting a direction inwhich variance of the signal values is smallest based on the pluralityof filtering results, directions in which signal values vary accordingto various variation modes. Accordingly, the direction in which varianceof the signal values is smallest may be detected more adequately, andnoise in an image may be reduced more accurately without blurring anedge or the like.

Further, in the signal processing apparatus according to the presentinvention, the signal classification means may be a means that sets thetarget pixel as a processing target of noise reduction only when thenumber of pixels classified into a group having a relatively smallsignal value within the predetermined range of area around the targetpixel exceeds a predetermined threshold value.

As for the “group having a relatively small signal value”, a groupclassified into a flat region may be one of the examples.

A noise reduction apparatus according to the present invention is anapparatus including an image noise reduction means that performs, basedon the classification result of the signal classification processingperformed by the signal processing apparatus of the present invention,noise reduction processing on each of the color component images.

In the noise reduction apparatus according to the present invention, thenoise reduction means may be a means that shifts the level of a signalvalue of a target pixel for processing such that a mean value of thesignal value of the target pixel and a signal value of each pixelincluded in a predetermined range of area around the target pixel ineach of the color component images corresponds to the position of originof a color space of each of the plurality of color components, performsa calculation of noise reduction on the shifted target pixel accordingto the level, and restores the level according to the amount of theshift.

Further, in the noise reduction apparatus according to the presentinvention, the noise reduction means may be a means that estimates theamount of noise in the target pixel, calculates a statistical valuerepresenting the amount of noise in the target pixel based on a signalvalue of the target pixel and a signal value of each pixel included in apredetermined range of area around the target pixel, compares theestimated amount of noise with the statistical value, and determineswhether to use the estimated amount of noise or the statistical value inthe calculation of noise reduction based on the comparison result.

Still further, in the noise reduction apparatus according to the presentinvention, the mean value may be a mean value of the signal values ofpixels classified into the group having a relatively small signal valuewithin the predetermined range of area.

A signal processing method according to the present invention is amethod including the steps of:

performing a first noise reduction processing on an image, in which eachof multitudes of pixels has one of a plurality of color components andthe color components are distributed regularly, based only on pixelarrangement to obtain a first processed image;

separating the first processed image into each of the color componentsto obtain a plurality of color component images; and

comparing a signal value of a target pixel for processing with a signalvalue of each pixel included in a predetermined range of area around thetarget pixel, and classifying each pixel within the predetermined rangeof area into one of a plurality of groups based on the comparisonresult.

A noise reduction method according to the present invention is a methodincluding the step of performing noise reduction processing, based onthe classification result of the signal classification processing of thesignal processing method according to the present invention, on each ofthe color component images.

Note that the program of the present invention may be provided beingrecorded on a computer readable medium. Those who are skilled in the artwould know that computer readable media are not limited to any specifictype of device, and include, but are not limited to: floppy disks, CD's,RAM's, ROM's, hard disks, magnetic tapes, and internet downloads, inwhich computer instructions can be stored and/or transmitted.Transmission of the computer instructions through a network or throughwireless transmission means is also within the scope of this invention.Additionally, computer instructions include, but are not limited to:source, object and executable code, and can be in any language includinghigher level languages, assembly language, and machine language.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of a digital camera to which asignal processing apparatus and noise reduction apparatus according to afirst embodiment of the present invention is applied, illustrating theconstruction thereof.

FIG. 2 illustrates a portion of the light receiving surface of a CCD(honeycomb arrangement).

FIG. 3 illustrates a portion of the light receiving surface of a CCD(Beyer arrangement).

FIG. 4 is a block diagram illustrating an electrical configuration ofthe image processing section.

FIG. 5 is a schematic block diagram of a noise reduction section in thefirst embodiment.

FIG. 6 is a flowchart illustrating processing performed in the firstembodiment.

FIGS. 7A to 7D illustrate pre-filtering processing on a CCD having anarray structure of honeycomb arrangement.

FIGS. 8A to 8D illustrate pre-filtering processing on a CCD having anarray structure of Beyer arrangement.

FIG. 9 illustrates separation of color components.

FIG. 10 is a flowchart of signal classifying processing.

FIGS. 11A and 11B illustrate signal classifying processing.

FIG. 12 is a flowchart of noise reduction processing.

FIG. 13 is a schematic block diagram of a noise reduction sectionaccording to a second embodiment, illustrating the construction thereof.

FIG. 14 is a conceptual diagram illustrating processing performed by agradient discrimination filtering processing section according to thesecond embodiment.

FIGS. 15A and 15B illustrate high-pass filtering.

FIG. 16 is a conceptual diagram illustrating processing performed by agradient discrimination filtering section according to a thirdembodiment.

FIG. 17 illustrates amplitude characteristics of a first derivativefilter and a second derivative filter.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, exemplary embodiments of the present invention will bedescribed with reference to the accompanying drawings. FIG. 1 is aschematic block diagram of a digital camera to which a signal processingapparatus and noise reduction apparatus according to a first embodimentof the present invention is applied, illustrating the constructionthereof. As shown in FIG. 1, the digital camera 1 includes an operationsystem 2 having an operation mode switch, a zoom-lever, an up-down andright-left button, a release button, a power switch, and the like, andan operation system control section 3, which is an interface fortransferring operational contents of the operation system 2 to a CUP 40.

An imaging system 6 includes a focus lens 10 a and a zoom lens 10 b thatconstitute a taking lens 10. The respective lenses are movable in theoptical axis directions by a focus lens drive section 11 and a zoom-lensdrive section 12 respectively, each of which including a motor and amotor driver. The focus lens drive section 11 controls movement of thefocus lens 10 a based on focus drive amount data outputted from an AFprocessing section 30. The zoom-lens drive section 12 controls movementof the zoom-lens 10 b based on data of operated amount of thezoom-lever.

An aperture diaphragm 14 is driven by an aperture diaphragm drivesection 15 that includes a motor and a motor driver. The aperturediaphragm drive section 15 controls the aperture diameter of theaperture diaphragm based on aperture value data outputted from an AE/AWBcontrol section 31.

A shutter 16 is a mechanical shutter, and is driven by a shutter drivesection 17 which includes a motor and a motor driver. The shutter drivesection 17 performs open/close control of the shutter 16 according to asignal generated by the depression of the release button and shutterspeed data outputted from the AE/AWE control section 31.

A CCD 18, which is an image sensor, is provided on the rear side of theoptical system. The CCD 18 has a photoelectric surface that includesmultitudes of light receiving elements disposed two-dimensionally, andthe light representing a subject image transmitted through the opticalsystem is focused on the photoelectric surface and subjected to aphotoelectric conversion.

FIG. 2 illustrates a portion of the light receiving surface of the CCD18. As shown in FIG. 2, the CCD 18 has an array structure of honeycombarrangement in which light receiving elements are arranged in acheckerboard pattern. For example, 4096 and 1540 light receivingelements are disposed in column and row directions respectively. Thus,if a subject is imaged using this CCD, image data representing a subjectimage with 4096 and 1540 pixels in the column and row directionsrespectively are obtained. The signal value obtained from each lightreceiving element corresponds to the signal value of each pixel of theimage.

In addition, a color filter array in which R, G, B filters are arrangedregularly is disposed on the light receiving surfaces of multitudes oflight receiving elements. As shown in FIG. 2, a color filter havingcharacteristics to pass any one of a red color component (R colorcomponent), a blue color component (B color component), and first andsecond green color components (the first and second green colorcomponents may have the same characteristics, G color component) isdisposed in the color filter array with respect to each light receivingelement. Filters that pass the red color component, blue colorcomponent, first green color component, and second green color componentare denoted by the alphabets R, B, Gr, and Gb respectively.

Also, as shown in FIG. 2, filters that pass red light component andfilters that pass blue light component are formed alternately on thelight receiving surfaces of the light receiving elements in the oddnumber columns. Filters that pass the first green light component andfilters that pass the second green light component are formedalternately on the light receiving surfaces of the light receivingelements in the even number columns.

As for the CCD 18, a CCD having an array structure of Beyer arrangementin which pixels are arranged in a square pattern as illustrated in FIG.3 may also be used, other than a CCD having an array structure ofhoneycomb arrangement as illustrated in FIG. 2. In the CCD having Beyerarrangement shown in FIG. 3, filters that pass red light component andfilters that pass the first green light component are formed alternatelyon the light receiving surfaces of the light receiving elements in theodd number columns. Filters that pass second green light component andfilters that pass the blue light component are formed alternately on thelight receiving surfaces of the light receiving elements in the evennumber columns.

The CCD 18 outputs charges stored in the respective pixels line by lineas serial analog image signals in synchronization with a verticaltransfer clock signal and a horizontal transfer clock signal suppliedfrom a CCD control section 19. The charge storage time of each pixel,that is, exposure time is determined by an electronic shutter drivesignal supplied from the CCD control section 19. The CCD 18 isgain-adjusted by the CCD control section 19 so that an analog imagesignal having a predetermined level is obtained.

The analog image signals picked up by the CCD 18 are inputted to ananalog signal processing section 20. The analog signal processingsection 20 includes: a correlated double sampling circuit (CDS) forremoving noise from the analog signals; an automatic gain controller(AGC) for controlling the gain of the analog signals; and an A/Dconverter (ADC) for converting the analog signals to digital signals.Hereinafter, processing performed by the analog signal processingsection 20 is referred to as the “analog signal processing”. The imagedata converted to digital signals are CCD-RAW data in which each pixelhas RGB density values. That is, in the CCD-RAW data, data having colorcomponents according to the color filters formed on the light receivingsurfaces of the light receiving elements serially appear line-by-line.Each pixel of a CCD-RAW image represented by the CCD-RAW data isrepresented by any one of the red color component, blue color component,first green color component, and second green color component, and doesnot have signal values of the other color components. The signal valuesof the other color components are interpolated by color interpolation tobe described later.

A timing generator 21 is a generator that generates timing signals,which are inputted to the shutter drive section 17, CCD control section19, and analog signal processing section 20, thereby the operation ofthe release button, open/close of the shutter 16, charge acquisition ofthe CCD 18, and the processing of the analog signal processing section20 are synchronized.

A flash control section 23 causes a flash 24 to emit light at the timeof photographing.

An image input controller 25 writes the CCD-RAW data, inputted from theanalog signal processing section 20, into a frame memory 26.

The frame memory 26 is a work memory used when various types of imageprocessing (signal processing), to be described later, are performed onthe image data, and constituted, for example, by a SDRAM (SynchronousDynamic Random Access Memory) that performs data transfer insynchronization with a bus clock signal having a constant frequency.

A display control section 27 is a control section for causing a liquidcrystal monitor 28 to display the image data stored in the frame memory26 as through images, or to display image data stored in the recordingmedium 35 when in playback mode.

The AF processing section 30 and AE/AWB processing section 31 determinean imaging condition based on a pre-image. The pre-image is an imagebased on the image data stored in the frame memory 26 as a result ofpre-imaging performed by the CCD 18, which is caused by the CPU 40 thathas detected a halfway depression signal generated when the releasebutton is depressed halfway.

The AF processing section 30 detects a focus position based on thepre-image, and outputs focus drive amount data (AF processing). As forthe focus position detection method, for example, a passive method thatmakes use of the fact that when a desired subject is focused, thecontrast becomes high.

The AE/AWB processing section 31 measures subject brightness based onthe pre-image, determines ISO sensitivity, aperture value, shutterspeed, and the like, and determines the ISO sensitivity data, aperturevalue data, and shutter speed data as an exposure setting value(AEcontrol), as well as automatically adjusting the white balance at thetime of photographing (AWB control).

FIG. 4 is a block diagram of the image processing section 32illustrating the electrical configuration thereof. As shown in FIG. 4,the image processing section 32 performs, on the CCD-RAW data of a finalimage, noise reduction, offset correction, gain correction, colorcorrection, gamma correction, and color interpolation for interpolatingeach color component of the CCD-RAW data in a noise reduction section50, an offset correction section 51, a gain correction section 52, acolor correction section 53, a gamma correction section 54, and a colorinterpolation section 55 respectively. Then, it performs, in a YCprocessing section 56, YC processing in which the CCD-RAW datainterpolated in the color component are converted to YC data constitutedby Y data (luminance signal data), Cb data (blue color difference signaldata), and Cr data (red color difference signal data).

The features of the first embodiment are in the processing performed inthe noise reduction section 50, but will be described later.

The term “final image” as used herein means an image based on image datapicked up by the CCD 18 through a main photographing performed when therelease button is fully depressed and stored in the frame memory 16 viathe analog signal processing section 20 and the image input controller25.

A compression/expansion section 33 generates an image file, byperforming, for example, JPEG format compression on the CCD-RWA data ofthe final image processed by the image processing section 32. A tag thatincludes auxiliary information, such as the date and time ofphotographing, and the like stored based on, for example, Exit format orthe like, is attached to the image file. Further, thecompression/expansion section 33 reads out a compressed image file fromthe recording medium 35 and performs expansion thereon when in playbackmode. The expanded image data are outputted to the monitor 28 and animage represented by the image data is displayed.

The media control section 34 gains access to the recording medium 35 tocontrol read/write operations of image files.

An internal memory 36 stores various constants to be set within thedigital camera 1, a program to be performed by the CPU 40, and the like.

The CPU 40 controls each section of the main body of the digital camera1 in response to the signals from various sections, including theoperation system 2, AF control section 30, and the like.

Various processing sections, frame memory 26, CPU 40, and the like areconnected to a data bus 41, and digital image data, variousinstructions, and the like are exchanged through the bus 41.

The digital camera 1 according to the first embodiment is constructed inthe manner as described above, and CCD-RAW data obtained by the CCD 18by photographing are processed by the image processing section 32, thenthe processed image data are compressed by the compression/expansionsection 33, and the compressed image data are recorded on the recordingmedium 35 by the media control section 34.

Noise reduction processing performed in the first embodiment will now bedescribed in detail. FIG. 5 is a schematic block diagram of the noisereduction section 50 in the first embodiment. Here, it is noted thatgreen color component of CCD-RAW data is constituted by the first andsecond color components, Gr and Gb, but in the following description thecolor components, Gr and Gb will be treated as the same color component.As shown in FIG. 5, the noise reduction section 50 includes: a firstcolor component separation section 61 that separates inputted CCD-RAWdata (R, G, B) into each of RGB color components to obtain colorcomponent images R, G, and B; a pre-filtering section 62 (a first noisereduction processing means) that performs pre-filtering processing(afirst noise reduction processing) on the CCD-RAW data (R, G, B); asecond color component separation section 63 that separates thepre-filtered CCD-RAW data into each of RGB color components to obtaincolor component images RL, GL, and BL; a signal classification section64 that performs signal classifying processing, as will be describedlater, on the color component images RL, GL, and BL; a processingsection 65 (an image noise reduction means) that performs noisereduction processing on the color component images R, G, and B based onthe classifying result performed by the signal classification section 64to obtain noise reduction processed color component images Rs, Gs, andBs; and a combining section 66 that generates noise reduction processedCCD-RAW data (CCD-RAW′) from the color component images Rs, Gs, and Bs.

Hereinafter, functions of the first color component separation section61, pre-filtering section 62, second color component separation section63; signal classification section 64; and processing section 65 will bedescribed based on the flowchart shown in FIG. 6. As described above,when a final photographing is performed and CCD-RAW data (R, G, B) areinputted to the noise reduction processing section 50 of the imageprocessing section 32, the processing is initiated. First, the firstcolor component separation section 61 separates the CCD-RAW data intoeach of RGB color components to obtain color component images R, G, andB (step ST1). In the mean time, the pre-filtering section 62 performspre-filtering processing on the CCD-RAW data (R, G, B) (step ST2).

The pre-filtering processing is filtering processing on a processingtarget pixel and pixels around the target pixel using a low-pass filterbased only on the pixel arrangement of the CCD-RAW data (R, G, B)without regarding color distribution of each pixel. Here, as for thelow-pass filter, for example, a low-pass filter that performs an averagecalculation or a weighted average calculation of the processing targetpixel and four pixels around the target pixel may be employed.

In a case where the CCD 18 has the array structure of honeycombarrangement shown in FIG. 2, and the color component of the target pixelis G, the pre-filtering processing is filtering processing using signalvalues of B color component pixels and signal values of R colorcomponent pixels located at the upper right, lower right, lower left,and upper left of the target pixel, as illustrated in FIGS. 7A and 7B.If the color component of the target pixel is R or B, then thepre-filtering processing is filtering processing using signal values ofG color component pixels located at the upper right, lower right, lowerleft, and upper left of the target pixel, as illustrated in FIGS. 7C and7D.

In the mean time, in a case where the CCD 18 has the array structure ofBeyer arrangement shown in FIG. 3, and the color component of the targetpixel is G, the pre-filtering processing is filtering processing usingsignal values of B color component pixels and signal values of R colorcomponent pixels located at the upper, lower, left, and right of thetarget pixel, as illustrated in FIGS. 8A and 8B. If the color componentof the target pixel is R or B, then the pre-filtering processing isfiltering processing using signal values of G color component pixelslocated at the upper, lower, left, and right of the target pixel, asillustrated in FIGS. 8C and 8D.

Pre-filtering processing when the CCD 18 has the honeycomb arrangementis shown in formulae (1) to (3) and pre-filtering processing when theCCD 18 has the Beyer arrangement is shown in formulae (4) to (6) below.In formulae (1) to (6), the suffix (0, 0) denotes the coordinates of thetarget pixel, and the suffix (i, j) (i, j=−1 to 1, i indicateshorizontal directions, and j indicates vertical directions) denotes thecoordinates of a pixel around the target pixel. In addition, “a” is thefilter coefficient of the low-pass filter. Further, formula (2)corresponds to the processing of FIG. 7B, and formula (5) corresponds tothe processing of FIG. 8B.

RL _(0,0)=(a _(−1,−1) *G _(−1,−1) +a _(−1,1) *G _(−1,1) +a _(1,−1) *G_(1,−1) +a _(1,1) *G _(1,1) +a _(0,0) *R _(0,0))/(a _(−1,−1) +a _(−1,1)+a _(1,−1) +a _(1,1) +a _(0,0))   (1)

GL _(0,0)=(a _(−1,−1) *R _(−1,−1) +a _(−1,1) *B _(−1,1) +a _(1,−1) *B_(1,−1) +a _(1,1) *R _(1,1) +a _(0,0) *G _(0,0))/(a _(−1,1) +a _(−1,1)+a _(1,−1) +a _(1,1) +a _(0,0))   (2)

BL _(0,0)=(a _(−1,−1) *G _(−1,−1) +a _(−1,1) *G _(−1,1) +a _(1,−1) *G_(1,−1) +a _(1,1) *G _(1,1) +a _(0,0) *B _(0,0))/(a _(−1,−1) +a _(−1,1)+a _(1,−1) +a _(1,1) +a _(0,0))   (3)

RL _(0,0)=(a _(−1,0) *G _(−1,0) +a _(0,−1) *G _(0,−1) +a _(1,0) *G_(1,0) +a _(0,1) *G _(0,1) +a _(0,0) *R _(0,0))/(a _(−1,0) +a _(0,−1) +a_(1,0) +a _(0,1) +a _(0,0))   (4)

GL _(0,0)=(a _(−1,0) *B _(−1,0) +a _(0,−1) *R _(0,−1) +a _(1,0) *B_(1,0) +a _(0,1) *R _(0,1) +a _(0,0) *G _(0,0))/(a _(−1,0) +a _(0,−1) +a_(1,0) +a _(0,1) +a _(0,0))   (5)

BL _(0,0)=(a _(−1,0) *G _(−1,0) +a _(0,−1) *G _(0,−1)+a_(1,0) *G _(1,0)+a _(0,1) *G _(0,1) +a _(0,0) *B _(0,0))/(a _(−1,0) +a _(0,−1) +a _(1,0)+a _(0,1) +a _(0,0)) (6)

By performing the pre-filtering in the manner as described above, randomnoise in the CCD-RAW data may be removed to a certain extent.

In the present embodiment, pre-filtering processing is performed on theCCD-RAW data prior to color interpolation processing, because noise inthe CCD-RAW data in this stage maintains spatial randomness. That is,after color interpolation processing, the randomness of noise in eachcolor component image is lost, so that noise can not be reduced bypre-filtering processing in this case. Therefore, in the presentembodiment, pre-filtering processing is performed before colorinterpolation processing.

Next, the second color component separation section 63 separates theCCD-RAW data (RL, GL, BL) into color component images RL, GL, and BL ofR, G, B color components respectively (step ST3).

FIG. 9 illustrates separation of the color components. In the presentembodiment, CCD-RAW data (RL, GL, BL) are separated such that the pixelarrangement of an image represented by the CCD-RAW data represents animage constituted only by each of RGB color components, thereby colorcomponent images RL, GL and BL are obtained.

Then, signal classifying processing is performed by the signalclassification section 64. In the signal classifying processing, a noisereduction processing target area BA of 9×9 pixels (81 pixels) in thecolumn directions and row directions respectively is set in each ofcolor component images RL, GL, and BL. Then, all of the pixels withinthe processing target area BA are classified into a flat region wherethe variation of signal values is small and a signal region where thevariation of signal value is large based on the signal value of eachpixel within the processing target area BA. It is noted that the size ofthe processing target area BA is not limited to the 81 pixels.

FIG. 10 is a flowchart of the signal classifying processing. Signalclassifying processing performed on each of the color component imagesPL, GL, and BL is identical, so that only the signal classifyingprocessing performed on the green color component image GL will bedescribed here. The target pixel for signal classifying processing isset on a first pixel (step ST11), and one pixel (discrimination targetpixel) (i,j) is set on a first pixel within the processing target areaBA of 9×9 pixels centered on the target pixel (step ST12), and adetermination is made as to whether or not the absolute difference|GL(i,j)−GL(5,5)| between the signal value GL(i,j) of the discriminationtarget pixel (i,j) and the signal value GL(5,5) of the target pixel(5,5) located in the center of the processing target area BA exceeds apredetermined threshold value Th1 (step ST13). The coordinates of theprocessing target area BA are set such that the upper left cornerthereof corresponds to (1,1). Here, the ratio between the signal valueGL(i,j) and signal value GL(5,5) may be used in place of the absolutedifference |GL(i,j)−GL(5,5)|.

If step ST13 is negative, the discrimination target pixel (i,j) isclassified into a flat region group having a relatively sam11 variationof signal values, since the discrimination target pixel(i,j) isconsidered to have a correlation with the target pixel(5,5) (step ST14),and the number “k” of the pixels classified into the flat region groupis incremented by 1 (step ST15). The initial value of “k” is set to 1since the target pixel (5, 5) serves as the reference.

On the other hand, if step ST13 is positive, the discrimination targetpixel (i,j) is classified into a signal region group having a largesignal value difference from the target pixel (5, 5), instead of theflat region group, since the discrimination target pixel (i,j) isconsidered not to have a correlation with the target pixel(5,5) (stepST16).

Following step ST15 or step ST16, a determination is made as to whetheror not the classifying is completed for all of the pixels within theprocessing target area BA (step ST17). If step ST17 is negative, thediscrimination target pixel is set on the next pixel (step ST18), andthe processing returns to step ST13 to repeat the processing from stepST13 onward.

This results in that the processing target area BA of 9×9=81 pixelsshown in FIG. 11A to be classified into the flat region enclosed by abold line with reference to the central target pixel (shown by hatchedlines) and the signal region as illustrated in FIG. 11B. The number ofpixels in the flat region shown in FIG. 11B is 56. It is noted that eachof the pixels classified into the flat region is denoted by a referencesymbol MG in the color component images RL, GL, and BL and in the colorcomponent images R, G, and B, in order to indicate that the pixelbelongs to the flat region. Further, in the color component images RLand BL, each pixel classified in to the flat region is denoted byreference symbols MR and MB respectively. Still further, in someinstances, the reference symbol MR, MG, or MB is used for the signalvalue of each pixel classified into the flat region.

Then, if step ST17 is positive, a determination is made whether or notthe number “k” of the pixels MG classified into the flat region groupwithin the processing target area BA exceeds a predetermined thresholdvalue Th2 (step ST19). If step ST17 is positive, the target pixels areset as the target of noise reduction processing in the subsequent stage(step ST20), and positions of the flat region pixels MG within theprocessing target area BA and “k” are outputted (step ST21). In the meantime, if step ST19 is negative, the target pixels are excluded from thetarget of noise reduction processing (step ST22).

Here, if the variation in the signal values is small within a processingtarget area BA around a target pixel, the value “k” will becomerelatively large and will exceed the threshold value Th2, which meansthat the target pixels are highly likely to be located in the flatregion where noise is quite visible, so that they are set as the targetof noise reduction processing. In contrast, if an edge or a fine patternis present within a processing target area BA around a target pixel, thevariation in the signal values will become great. Consequently the value“k” will become relatively small and will not exceed the threshold valueTh2. In such a case, the target pixels are excluded from the target ofnoise reduction processing, since noise reduction processing on suchpixels causes the edge or fine pattern to be blurred.

Following step ST21 or ST22, a determination is made as to whether ornot the classifying is completed for all of the pixels of the colorcomponent image GL (step ST23). If step ST23 is negative, the targetpixel is set on the next pixel (step ST24), and the processing returnsto step ST12 to repeat the processing from step ST12 onward.

Returning to FIG. 6, following the signal classifying processing, theprocessing section 65 performs noise reduction processing on the colorcomponent images R, G, and B (step ST5). FIG. 12 is a flowchart of thenoise reduction processing. Noise reduction processing performed on eachof the color component images R, G, and B is identical, so that only thesignal classifying processing performed on the green color componentimage C will be described here. The target pixel for noise reductionprocessing is set on a first pixel (step ST31), and a determination ismade as to whether or not the target pixel is set as a target of noisereduction processing by the signal classifying processing describedabove (step ST32). If step ST32 is negative, the processing proceeds tostep ST39 to be described later.

If step ST32 is positive, a mean value MGm of signal values of pixels MGclassified into the flat region within a processing target area BAhaving the same size as that of the signal classifying processing iscalculated by formulae (7) and (8) shown below (step ST33). In addition,a variance value (statistical value representing noise) σ² _(MG)representing a noise amount based on the signal value within theprocessing target area BA is calculated (step ST34).

$\begin{matrix}{{MGm} = {\frac{1}{\sum{\sum{wij}}}{\sum{\sum{{wij} \cdot {{MG}\left( {i,j} \right)}}}}}} & (7) \\{\sigma_{MG}^{2} = {\frac{1}{k - 1}{\sum{\sum\left( {{{MG}\left( {i,j} \right)} - {MGm}} \right)}}}} & (8)\end{matrix}$

In formula (7), wij is the weighting factor for the signal valueMG(i,j). In formulae (7) and (8), Σ indicates that “k” signal valuesMG(i,j) are all added up. Further, formulae (7) and (8) calculate themean value and variance value of the color component image G, and not ofthe color component image GL.

In the mean time, the processing section 65 estimates the noise amountwithin the processing target area BA using the mean value MGm calculatedin step ST33 (step ST35). More specifically, the processing section 65calculates the noise variance value σ² _(nMG) representing the estimatedamount of noise, as indicated in formula (9) below. In formula (9), A,B, and C are coefficients determined by the dark noise, optical shotnoise and fixed pattern noise which are inherent to the CCD 18. The term“offset” in formula (9) means the offset value of the CCD 18. It isnoted that the variance value σ² _(MG) calculated by formula (8) ishereinafter referred to as “signal variance value” in order todistinguish from the noise variance value σ² _(nMG) calculated byformula (9).

σ² _(nMG) =A·(MGm−offset)² +B·(MGm−offset)+C   (9)

Then, the processing section 65 determines whether or not the ratio ofthe noise variance value σ² _(nMG) to the signal variance value σ²_(MG), σ² _(MG)/σ² _(nMG) is smaller than a predetermined thresholdvalue Th3 (step ST36). Here, if the signal classifying processingdescribed above is performed appropriately, the signal variance value σ²_(MG) is substantially equal to the noise variance value σ² _(nMG), thusthe ratio of σ² _(MG)/σ² _(nMG) takes a value close to 1. In the meantime, the signal classifying processing uses color component images RL,GL, and BL reduced in noise to a certain extent by the pre-filteringprocessing. Consequently, even when a target pixel is classified intothe flat region, an edge or a fine pattern is possibly included in thecolor component images R, G, and B not subjected to the pre-filteringprocessing. In such a case, the signal variance value σ² _(MG) becomesgreater than the noise variance value σ² _(nMG), thus the ratio of σ²_(MG)/σ² _(nMG) becomes great.

If step ST36 is positive, it can be regarded that the target pixel islocated in the flat region, so that noise in the target pixel is reducedby substituting the signal variance value σ² _(MG) to σ² _(XG) informula (10) below (ST37). In the mean time, if step ST36 is negative,it is highly likely that the target pixel is located in the signalregion, so that noise in the target pixel is reduced by substituting thenoise variance value σ² _(nMG) to σ² _(XG) in formula (10) below (stepST38). In formula (10), a is the coefficient for determining noisereduction level, the value of which is from 0 to 1.

$\begin{matrix}{{Gs} = {{MGm} + {\left( \frac{\sigma_{MG}^{2} - {\alpha \cdot \sigma_{XG}^{2}}}{\sigma_{MG}^{2}} \right) \cdot \left( {G - {MGm}} \right)}}} & (10)\end{matrix}$

In formula (10), by subtracting the mean value MGm from the pixel valueG of the target pixel, the signal value of the pixel within theprocessing target area BA is shifted to the position of origin of thecolor space having G color component in the coordinate system, and theshifted target pixel is multiplied by (σ² _(MG)−α·σ² _(XG))/σ² _(MG).Then, by adding the mean value MGm after the multiplication, the signalvalue of the target pixel is restored to a level corresponding to theoriginal signal value. Here, (σ² _(MG)−α·σ² _(XG))/σ² _(MG) takes avalue in the range from 0 to 1, so that the pixel value Gs of theprocessed target pixel takes a value in the range from the pixel value Gprior to the processing and the mean value MGm.

Accordingly, for example, if α=1 and the ratio of σ² _(MG)/σ² _(nMG) issmaller than the predetermined threshold value Th3, then σ² _(XG)=signalvariance value σ² _(MG), thus (σ² _(MG)−α·σ² _(XG))/σ² _(MG) becomes 0and the signal value Gs of the processed target pixel becomes equal tothe mean value MGm. If the ratio of σ² _(MG)/σ² _(nMG) is greater thanor equal to the predetermined threshold value Th3, then σ² _(XG)=noisevariance value σ² _(nMG), thus σ² _(MG)>>σ² _(nMG) and (σ² _(MG)−α·σ²_(XG))/σ² _(MG) takes a value which is closer to 1, so that the signalvalue Gs of the processed target pixel takes a value which is closer tothe signal value prior to the processing.

Then, a determination is made as to whether or not the noise reductionprocessing is completed for all of the pixels (step ST39). If step ST39is negative, the target pixel is set on the next pixel (step ST40), andthe processing returns to step ST32 to repeat the processing from stepST32 onward. If step ST39 is positive, the noise reduction processing isterminated.

Returning to FIG. 6, the combining section 66 combines the processedcolor component images Rs, Gs, and Bs to generate processed CCD-RAW data(CCD-RAW′), thereby the noise reduction processing is completed. Thenoise-reduction processed CCD-RAW data are subjected to the offsetcorrection, color correction, gamma correction, and color interpolationfor interpolating each color component of the CCD-RAW data in the offsetcorrection section 51, gain correction section 52, color correctionsection 53, gamma correction section 54, and color interpolation section55 respectively. Then, in the YC processing section 56, the CCD-RAW datainterpolated in the color component are converted to YC data constitutedby Y data which are luminance signal data, Cb data which are blue colordifference signal data, and Cr data which are red color differencesignal data.

Thereafter, the compression/expansion processing section 33 performscompression on the CCD-RAW data processed by the image processingsection 32 in JPEG compression format or the like to generate an imagefile. The generated image file is recorded on the recording medium 35 bythe media control section 34.

In this way, in the first embodiment, pre-filtering processing isperformed on CCD-RAW data (R, G, B) to reduce noise to a certain extentwhen performing signal separation processing. Here, it is also possibleto classify the pixels within the processing target area BA into theflat region and the signal region without performing pre-filteringprocessing. However, if a large amount of noise is contained in aCCD-RAW image represented by the CCD-RAW data, a comparison resultbetween the signal value of a processing target pixel and the signalvalue of each pixel within a processing target area in each of the colorcomponent images R, G, and B does not show any appreciable differencedue to the noise, so that the pixels within the processing target areacan not be accurately classified into the flat region and signal region.In particular, a CCD-RAW image obtained by high sensitivityphotographing contains a great amount of noise, which makes theclassifying of pixels into the flat region and signal region even moredifficult.

In the first embodiment, pre-filtering processing is performed onCCD-RAW data, so that even if a large amount of noise is contained inthe CCD-RAW data, like those obtained by high sensitivity photographing,noise contained in color component images R, G, and B obtained from theCCD-RAW data may be reduced. Accordingly, pixels within a processingtarget area BA may be accurately classified into the flat region andsignal region.

Further, the pre-filtering processing is performed on CCD-RAW data basedonly on the pixel arrangement, i.e., without regarding colordistribution of each pixel, so that noise is reduced, in effect, in thefrequency band greater than or equal to Nyquist frequency of each of thecolor components included in an image. Consequently, the pre-filteredimage is slightly blurred, but the blur is reduced in the colorcomponent images RL, GL, and BL obtained by separating the image intoeach of the color components. This allows pixels within a processingtarget area BA to be classified into the flat region and signal regionaccording to the comparison result without being affected by the blur.

In the mean time, signal variation does not occur or is very small inthe flat region of an image, and if a target pixel is classified intothe flat region, the signal variance value σ² _(MG) represents thevariance of noise, so that, ideally, the signal variance value σ² _(MG)corresponds to the noise variance value σ² _(nMG). However, the σ²_(nMG) is an estimated value of noise amount calculated by formula (9)above, whereas the signal variance value σ² _(MG) is a variance valuecalculated for a certain limited area on an image based on actual signalvalues. Therefore, there may be a case where a portion where the signalvariance value σ² _(MG) and noise variance value σ² _(nMG) do notcorrespond to each other is developed on an image. In such a case, ifnoise reduction processing is performed based on the calculation offormula (10) using only the noise variance value σ² _(nMG), spots maypossibly may possibly be be developed in the flat region of the imagedue to the portion where the signal variance value σ² _(MG) and noisevariance value σ² _(nMG) do not correspond to each other.

In the mean time, the signal classifying processing described above usescolor component images RL, GL, and BL reduced in noise to a certainextent by the pre-filtering processing. Consequently, even when a targetpixel is classified into the flat region, an edge or a fine pattern ispossibly included in the color component images R, G, and B notsubjected to the pre-filtering processing. In such a case, the signalvariance value σ² _(MG) becomes great, so that if a calculation offormula (10) is made using only the signal variance value σ² _(MG), theedge or fine pattern may possibly be blurred.

Here, if the signal variance value σ² _(MG) substantially corresponds tothe noise variance value σ² _(nMG), i.e., if the estimated noise amountsubstantially corresponds to measured noise amount, the ratio of σ²_(MG)/σ² _(nMG) takes a value close to 1. In this case, in the firstembodiment, the target pixel is assumed to be located in the flatregion, and the calculation of formula (10) is made using the measurednoise amount, i.e., signal variance value σ² _(MG), so that noise in theflat region may be reduced appropriately without developing spots. Inthe mean time, if noise is overly reduced by the pre-filteringprocessing, the signal variance value σ² _(MG) becomes greater than thenoise variance value σ² _(nMG), so that the ratio of σ² _(MG)/σ² _(nMG)becomes large. In this case, the ratio between the signal variance valueσ² _(MG) and noise variance value σ² _(nMG), σ² _(MG)/σ² _(nMG) isgreater than or equal to the threshold value Th3, and the target pixelis assumed to be located in the signal region, substitution of σ² _(nMG)to the σ² _(XG) in formula (10) may prevent an edge or a fine pattern tobe blurred. Accordingly, noise may be reduced without causing the edgeor fine pattern to be blurred or spots to be developed in the flatregion.

Further, in the signal classifying processing, pixels classified intothe signal region are excluded from the target of noise reductionprocessing, so that noise may be reduced without causing an edge or afine pattern to be blurred.

In the first embodiment, a determination is made as to whether or notthe absolute difference |GL(i,j)−GL(5,5)| between the signal valueGL(i,j) of the discrimination target pixel (i,j) and the signal valueGL(5,5) of the central target pixel (5,5) exceeds a predetermined regiondiscrimination threshold value Th1. But, for example, an alternativemethod may be used in which two threshold values Th1 and Th1′(Th1>Th1′), and a determination is made as to whether the absolutedifference |GL(i,j)−GL(5,5)| is smaller than Th1′, greater than or equalto Th1′ and smaller than Th1, or greater than Th1, and thediscrimination target pixel GL(i,j) is classified into a plurality ofgroups. More specifically, if |GL(i,j)−GL(5,5)|<Th1′, the discriminationtarget pixel GL(i,j) is classified into the flat region group, ifTh1′≦|GL(i,j)−GL(5,5)|<Th1, the target pixel GL(i,j) is classified intoan intermediate group between the flat region and signal region groups,and if Th1<|GL(i,j)−GL(5,5)|, the target pixel GL(i,j) is classifiedinto the signal region group. In this case, each pixel classified intothe intermediate group is assumed to correspond to one-half the pixelclassified into the flat region in terms of counting, and may be addedto the number “k” of the pixels classified into the flat region.

Further, in the first embodiment, all of the pixels of the colorcomponent images RL, GL, and BL are classified first in the signalclassification section 64, and noise reduction processing is performedafter that in the processing section 65. Alternatively, signalclassifying result in the signal classification section 64 may besequentially outputted, and noise reduction processing may besequentially performed on the target pixels in the processing section65.

Next, a second embodiment of the present invention will be described. Inthe second embodiment, only the configuration of the noise reductionsection is different from that of the first embodiment. Therefore, onlythe difference from the first embodiment will be described hereinafter.FIG. 13 is a schematic block diagram of a noise reduction sectionaccording to the second embodiment, illustrating the constructionthereof. In FIG. 13, components identical to those in FIG. 5 are giventhe same reference numerals, and will not be elaborated upon furtherhere. The noise reduction section 50A according to the second embodimentdiffers from the noise reduction section 50 according to the firstembodiment, in that it has a gradient discrimination filtering section67 between the second color separation section 63 and signalclassification section 64.

The gradient discrimination filtering section 67 receives colorcomponent images RL, GL, and BL outputted from the second colorcomponent separation section 63, and detects a direction in whichvariance of the signal values is smallest of the four directions ofhorizontal (H direction), vertical (v direction), upper right to lowerleft (NE direction), and upper left to lower right (NW direction) withrespect to each pixel of the color component images RL, GL, and BL, andperforms filtering processing on the direction in which variance of thesignal values is smallest using a low-pass filter.

FIG. 14 is a conceptual diagram illustrating processing performed by agradient discrimination filtering section 67 in the second embodiment.As illustrated in FIG. 14, the gradient discrimination filtering section67 performs filtering processing on each pixel of the color componentimages RL, GL, and BL by high-pass filtering (H-pass) sections 70A to70D in the H, V, NE and NW directions respectively, and calculates theirabsolute values by absolute value calculation sections (abs) 71A to 71Das evaluation values Q_H, Q_V, Q_NE, and Q_NW respectively.

Here, if the CCD 18 is a CCD having the honeycomb arrangement shown inFIG. 2, the filtering processing by the high-pass filters (high-passfiltering processing) is filtering processing using each pixel adjacentto the target pixel located in the center, as shown in FIG. 15A. If theCCD 18 is a CCD having the Beyer arrangement shown in FIG. 3, thefiltering processing by the high-pass filters is filtering processingusing each pixel adjacent to the target pixel, as shown in FIG. 15B.

The high-pass filtering processing for the honeycomb arrangement CCD 18and for the Beyer arrangement CCD 18 is shown in formulae (11) to (14),and formulae (15) to (18) below respectively. In formulae (11) to (14),the suffix (0,0) denotes the coordinates of the target pixel, and thesuffix (i,j) (i,j=−2 to 2, i indicates horizontal directions, and jindicates vertical directions) denotes the coordinates of a pixel aroundthe target pixel. In addition, “b” is the filter coefficient of thehigh-pass filter. As for the high-pass filter, for example, a firstderivative filter of (b⁻¹, b₀, b₁)=(−1, 0, 1) or a second derivativefilter of (b⁻¹, b₀, b₁)=(−1, 2, −1). Here, the description will be madeonly for green color component, since red and blue color componentsmaybe processed in the same manner as the green color component.

Q _(—) H=|b ⁻¹ *GL _(−2,0) +b ₀ *GL _(0,0) +b ₁ *GL _(2,0))   (11)

Q _(—) V=|b ⁻¹ *GL _(0,−2) +b ₀ *GL _(0,0) +b ₁ *GL _(0,2))   (12)

Q _(—) NE=|b ⁻¹ *GL _(−1,1) +b ₀ *GL _(0,0) +b ₁ *GL _(1,1))   (13)

Q _(—) NW=|b ⁻¹ *GL _(−1,−1) +b ₀ *GL _(0,0) +b ₁ *GL _(1,1))   (14)

Q _(—) H=|b ⁻¹ *GL _(−1,0) +b ₀ *GL _(0,0) +b ₁ *GL _(1,0))   (15)

Q _(—) V=|b ⁻¹ *GL _(0,−0) +b ₀ *GL _(0,0) +b ₁ *GL _(0,1))   (16)

Q _(—) NE=|b ⁻¹ *GL _(−1,1) +b ₀ *GL _(0,0) +b ₁ *GL _(1,−1))   (17)

Q _(—) NW=|b ⁻¹ *GL _(−1,−1) +b ₀ *GL _(0,0) +b ₁ *GL _(1,1))   (18)

Then, a direction where the smallest value of the evaluation values Q_H,Q_V, Q_NE, and Q_NW is calculated is determined in a selector 72 to bethe direction in which variance of the signal values is smallest, andfiltering processing by a low-pass filter is performed on the directionin which variance of the signal values is smallest by a low-passfiltering (L-pass) section 73 to output processed color component imagesRLP, GLP, and BLP. Following formulae (19) to (22) represent low-passfiltering in the H, V, NE and NW directions respectively when the CCD 18has the honeycomb arrangement, and formulae (23) to (26) representlow-pass filtering in the H, V, NE and NW directions respectively whenthe CCD 18 has the Beyer arrangement.

GLP _(0,0) _(—) H=(c ⁻¹ *GL _(−2,0) +C ₀ *GL _(0,0) +c ₁ *GL_(2,0)+1.5)/3   (19)

GLP _(0,0) _(—) V=(c ⁻¹ *GL _(0,−2) +C ₀ *GL _(0,0) +c ₁ *GL_(0,2)+1.5)/3   (20)

GLP _(0,0) _(—) NE=(c ⁻¹ *GL _(−1,1) +C ₀ *GL _(0,0) +c ₁ *GL_(1,−1)+1.5)/3   (21)

GLP _(0,0) _(—) NW=(c ⁻¹ *GL _(−1,−1) +C ₀ *GL _(0,0) +c ₁ *GL_(1,1)+1.5)/3   (22)

GLP _(0,0) _(—) H=(c ⁻¹ *GL _(−1,0) +C ₀ *GL _(0,0) +c ₁ *GL_(1,0)+1.5)/3   (23)

GLP _(0,0) _(—) V=(c ⁻¹ *GL _(0,−1) +C ₀ *GL _(0,0) +c ₁ *GL_(0,1)+1.5)/3   (24)

GLP _(0,0) _(—) NE=(c ⁻¹ *GL _(−1,1) +C ₀ *GL _(0,0) +c ₁ *GL_(1,−1)+1.5)/3   (25)

GLP _(0,0) _(—) NW=(c ⁻¹ *GL _(−1,−1) +C ₀ *GL _(0,0) +c ₁ *GL_(1,1)+1.5)/3   (22)

In formulae (19) to (26), the reason for the addition of the value of1.5 is to give the round-off effect when divided by the value of 3.Further, in formulae (19) to (26), “c” is the filter coefficient of thelow-pass filter, and, for example, (c⁻¹, c₀, c₁)=(1, 1, 1) is used.

The color component images RLP, GLP, and BLP obtained by the gradientdiscrimination filtering section 67 in the manner as described above areinputted to the signal classification section 64, and the signalclassifying processing and noise reduction processing are performedthereon as in the first embodiment.

Here, as described above, noise in CCD-RAW data may be reduced to acertain extent by performing pre-filtering processing. However, if alarge amount of noise is contained in CCD-RAW data, as in particularlyhigh sensitivity photographing, the noise can not be reducedsatisfactorily by the pre-filtering processing. As a result, the signalclassifying processing can not be performed accurately. In this case, itis conceivable to increase the blur level when performing thepre-filtering processing, but if the blur level is increased, an edge orthe like included in the CCD-RAW data is also blurred, so that thesignal classifying processing can not be performed accurately after all.

In the second embodiment, a direction in which variance of the signalvalues is smallest is detected with respect to the color componentimages RL, GL, and BL, and filtering processing is performed on thedirection in which variance of the signal values is smallest by alow-pass filter. This may cause the color component images RL, GL, andBL to be further blurred, but the direction to be blurred does not crossthe edge, so that only the noise may be reduced without blurring theedge. Accordingly, only noise may be reduced without blurring edges, sothat even when high sensitivity photographing is performed, signalclassifying processing required for the subsequent noise reductionprocessing may be performed accurately.

Next, a third embodiment of the present invention will be described. Inthe third embodiment, only the configuration of the gradientdiscrimination filtering section is different from that of the secondembodiment. Therefore, only the difference from the second embodimentwill be described hereinafter.

Whereas the gradient discrimination filtering section 67 according tothe second embodiment uses only one type of high-pass filter forcalculating each of the gradient evaluation values Q_H, Q_V, Q_NE, andQ_NW, a gradient discrimination filtering section 68 according to thethird embodiment uses two types of high-pass filters having differentfrequency characteristics for calculating each of the gradientevaluation values Q_H, Q_V, Q_NE, and Q_NW.

FIG. 16 is a conceptual diagram illustrating processing performed by thegradient discrimination filtering section 68 according to the thirdembodiment. As illustrated in FIG. 16, the gradient discriminationfiltering section 68 performs filtering processing on each pixel of thecolor component images RL, GL, and BL by first high-pass filtering(H-pass1) sections 75A to 75D in the H, V, NE, and NW directionsrespectively, and calculates their absolute values by absolute valuecalculation sections (abs) 76A to 76D as evaluation values Q_H1, Q_V1,Q_NE1, and Q_NW1 respectively. Further, it performs filtering processingon each pixel of the color component images RL, GL, and BL by secondhigh-pass filtering (H-pass2) sections 77A to 77D in the H, V, NE and NWdirections respectively, and calculates their absolute values byabsolute value calculation sections (abs) 78A to 78D as evaluationvalues Q_H2, Q_V2, Q_NE2, and Q_NW2 respectively.

Then, the first gradient evaluation values Q_H1, Q_V1, Q_NE1, and Q_NW1and second gradient evaluation values Q_H2, Q_V2, Q_NE2, and Q_NW2 areweight added by adder sections 79A to 79D respectively, as shown informulae (27) to (30).

Q _(—) H=(w*Q _(—) H1+(1−w)*Q _(—) H2+1)/2   (27)

Q _(—) V=(w*Q _(—) V1+(1−w)*Q _(—) V2+1)/2   (28)

Q _(—) NE=(w*Q _(—) NE1+(1−w)*Q _(—) NE2+1)/2   (29)

Q _(—) NW=(w*Q _(—) NW1+(1−w)*Q _(—) NW2+1)/2   (30)

Here, the reason for the addition of the value of 1 is to give theround-off effect when divided by the value of 2.

Then, a direction in which a smallest value of the evaluation valuesQ_H, Q_V, Q_NE, and Q_NW is calculated is determined in a selector 80 tobe the direction in which variance of the signal values is smallest, andfiltering processing by a low-pass filter is performed on the directionin which variance of the signal values is smallest by a low-passfiltering (L-pass) section 81 to output processed color component imagesRLP, GLP, and BLP, as in the second embodiment.

Here, as for the first high-pass filter, for example, a first derivativefilter having a filter coefficient of (b⁻¹, b₀, b₁)=(−1, 0, 1) may beused, and as for the second high-pass filter, for example, a secondderivative filter having a filter coefficient of (b⁻¹, b₀, b₁)=(−1, 2,−1) may be used. FIG. 17 illustrates amplitude characteristics of thefirst and second derivative filters. It is noted that the frequency onthe horizontal axis in FIG. 17 is the frequency normalized by thesampling frequency. As illustrated in FIG. 17, comparison of amplitudecharacteristics between the first and second derivative filters showsthat the first derivative filter may detect a signal value gradient likea gradation, but it is not suitable for the detection of a gradient likea fine pattern. Whereas, the second derivative filter is not suitablefor the detection of a gradient like a gradation, but it is suitable forthe detection of a gradation like a fine pattern.

In the third embodiment, the gradient evaluation values Q_H1, Q_V1,Q_NE1, and Q_NW1 calculated using the first derivative filters andsecond gradient evaluation values Q_H2, Q_V2, Q_NE2, and Q_NW2calculated using the second derivative filters are weight added, so thatgradient variations that can be detected may be increased. Accordingly,various different variations in the signal values included in the colorcomponent images RL, GL, and BL may be detected.

In the third embodiment, the gradient evaluation values Q_H1, Q_V1,Q_NE1, and Q_NW1 obtained by the first derivative filters and secondgradient evaluation values Q_H2, Q_V2, Q_NE2, and Q_NW2 obtained by thesecond derivative filters are weight added. Alternatively, a greatervalue between each of the gradient evaluation values Q_H1, Q_V1, Q_NE1,and Q_NW1 obtained by the first derivative filters and each of thesecond gradient evaluation values Q_H2, Q_V2, Q_NE2, and Q_NW2 obtainedby the second derivative filters may be used as the final evaluationvalue.

In the first to third embodiment described above, the processing shownin the flowchart of FIG. 12 is performed as the noise reductionprocessing. But the noise reduction processing is not limited to thisand any processing may be used as long as it uses signal classifyingprocessing result. For example, the noise reduction processing may beperformed through filtering processing using a low-pass filter on atarget pixel classified into the flat region.

Further, in the first to third embodiment, a determination is made as towhether to use signal variance value σ² _(MG) or noise variance value σ²_(nMG) in formula (10) according to the ratio of the noise variancevalue σ² _(nMG) to the signal variance value σ² _(MG), σ² _(MG)/σ²_(nMG) in the noise reduction processing. Alternatively, thedetermination may be made according to a difference or an absolutedifference between the signal variance value σ² _(MG) and noise variancevalue σ² _(nMG).

As described above, embodiments of the signal classifying apparatus andnoise reduction apparatus are applied to digital cameras. Alternatively,they may be provided as stand-alone apparatuses that respectivelyperform signal classifying processing and noise reduction processing onCCD-RAW data obtained by a digital camera in the same manner asdescribed above. A program for causing a computer to function as themeans corresponding to the pre-filtering section 62, color componentseparation section 63, signal classification section 64, and processingsection 65, thereby causing the computer to perform processing like thatillustrated in FIGS. 6, 10, and 12 is another embodiment of the presentinvention. Further, a computer readable recording medium on which suchprogram is recorded is still another embodiment of the presentinvention.

1. A signal processing apparatus, comprising: a first noise reductionprocessing means that performs first noise reduction processing on animage, in which each of multitudes of pixels has one of a plurality ofcolor components and the color components are distributed regularly,based only on pixel arrangement to obtain a first processed image; acolor component separation means that separates the first processedimage into each of the color components to obtain a plurality of colorcomponent images; and a signal classification means that compares asignal value of a target pixel for processing with a signal value ofeach pixel included in a predetermined range of area around the targetpixel and classifies each pixel within the predetermined range of areainto one of a plurality of groups based on the comparison result.
 2. Thesignal processing apparatus according to claim 1, wherein the signalclassification means is a means that classifies each pixel within thepredetermined range of area into a flat region group having a relativelysmall variance of the signal values or a signal region group having arelatively large variance of the signal values according to thecomparison result.
 3. The signal processing apparatus according to claim1, wherein the first noise reduction processing is filtering processingby a low-pass filter on each of the pixels in the image.
 4. The signalprocessing apparatus according to claim 1, wherein: the apparatusfurther comprises a second noise reduction processing means thatperforms second noise reduction processing on each of the colorcomponent images according to a variance direction of the signal valuesin each of the pixels in each of the color component images; and thesignal classification means is a means that performs the signalclassification processing on each of the color component imagessubjected to the second noise reduction processing.
 5. The signalprocessing apparatus according to claim 4, wherein the second noisereduction processing means is a means that performs the second noisereduction processing by performing filtering processing on each of thepixels in each of the color component images in a plurality ofpredetermined directions by a high-pass filter to detect a direction inwhich the variance of the signal values is smallest based on thefiltering result, and performing filtering by a low-pass filter on thedirection in which the variance of the signal values is smallest.
 6. Thesignal processing apparatus according to claim 4, wherein the secondnoise reduction processing means is a means that performs the secondnoise reduction processing by performing filtering processing by aplurality of types of high-pass filters having different frequencycharacteristics from each other on each of the pixels in each of thecolor component images in a plurality of predetermined directions toobtain a plurality of filtering results, detecting a direction in whichthe variance of the signal values is smallest based on the plurality offiltering results, and performing filtering by a low-pass filter on thedirection in which the variance of the signal values is smallest.
 7. Thesignal processing apparatus according to claim 1, wherein the signalclassification means is a means that sets the target pixel as aprocessing target of noise reduction only when the number of pixelsclassified into a group having a relatively small signal values withinthe predetermined range of area around the target pixel exceeds apredetermined threshold value.
 8. A noise reduction apparatus,comprising an image noise reduction means that performs noise reductionprocessing, based on the classification result of the signalclassification processing performed by the signal processing apparatusaccording to claim 1, on each of the color component images.
 9. Thenoise reduction apparatus according to claim 8, wherein the noisereduction means is a means that shifts the level of a signal value of atarget pixel for processing such that a mean value of the signal valueof the target pixel and a signal value of each pixel included in apredetermined range of area around the target pixel in each of the colorcomponent images corresponds to the position of origin of a color spaceof each of the plurality of color components, performs a calculation ofnoise reduction on the shifted target pixel according to the level, andrestores the level according to the amount of the shift.
 10. The noisereduction apparatus according to claim 9, wherein the noise reductionmeans is a means that estimates the amount of noise in the target pixel,calculates a statistical value representing the amount of noise in thetarget pixel based on a signal value of the target pixel and a signalvalue of each pixel included in a predetermined range of area around thetarget pixel, compares the estimated amount of noise with thestatistical value, and determines whether to use the estimated amount ofnoise or the statistical value in the calculation of noise reductionbased on the comparison result.
 11. The noise reduction apparatusaccording to claim 9, wherein the mean value is a mean value of thesignal values of pixels classified into the group having a relativelysmall signal value within the predetermined range of area.
 12. The noisereduction apparatus according to claim 8, wherein the noise reductionmeans is a means that performs noise reduction processing on the targetpixel only when the number of pixels classified into the group having arelatively small signal value within the predetermined range of areaaround the target pixel exceeds a predetermined threshold value.
 13. Asignal processing method, comprising the steps of: performing a firstnoise reduction processing on an image, in which each of multitudes ofpixels has one of a plurality of color components and the colorcomponents are distributed regularly, based only on pixel arrangement toobtain a first processed image; separating the first processed imageinto each of the color components to obtain a plurality of colorcomponent images; and comparing a signal value of a target pixel forprocessing with a signal value of each pixel included in a predeterminedrange of area around the target pixel, and classifying each pixel withinthe predetermined range of area into one of a plurality of groups basedon the comparison result.
 14. A noise reduction method, comprising thestep of performing noise reduction processing, based on theclassification result of the signal classification processing of thesignal processing method according to claim 13, on each of the colorcomponent images.
 15. A computer readable recording medium on which aprogram for causing a computer to perform a signal processing method isrecorded, the method comprising the steps of: performing a first noisereduction processing on an image, in which each of multitudes of pixelshas one of a plurality of color components and the color components aredistributed regularly, based only on pixel arrangement to obtain a firstprocessed image; separating the first processed image into each of thecolor components to obtain a plurality of color component images; andcomparing a signal value of a target pixel for processing with a signalvalue of each pixel included in a predetermined range of area around thetarget pixel, and classifying each pixel within the predetermined rangeof area into one of a plurality of groups based on the comparisonresult.
 16. A computer readable recording medium on which a program forcausing a computer to perform a noise reduction method is recorded, themethod comprising the step of performing noise reduction processing,based on the classification result of the signal classificationprocessing of the signal processing method according to claim 13, oneach of the color component images.